fix(cdc): mark all leaves dirty when a TRUNCATE is decoded#145
fix(cdc): mark all leaves dirty when a TRUNCATE is decoded#145mason-sharp wants to merge 2 commits into
Conversation
📝 WalkthroughWalkthroughChangesTRUNCATE Merkle synchronization
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 15 |
| Duplication | 3 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
91c1f4d to
2e79ea7
Compare
The CDC drain's message switch only handled Insert/Update/Delete, so the TruncateMessage pgoutput publishes for a TRUNCATE (the publication is created with the default publish list, which includes truncate) was silently dropped. No leaves were dirtied, the truncated node's Merkle tree kept its pre-truncate hashes, and mtree table-diff reported the nodes as in sync. A truncate carries only relation OIDs -- no per-row changes -- so per-PK tracking is impossible. React with the adaptive drain's bulk path instead: mark every leaf of each truncated relation dirty (cascades list all published relations) so the next tree update rehashes from the live, now-empty table. Buffered UPDATEs for the table are purged as at escalation time; a failed mark is terminal so a stale tree is never reported current. Applies to both bounded drains (table-diff/update) and continuous listen, which share the decode switch. The integration test reproduces the field report: identical nodes, clean diff, TRUNCATE on n2, then a second diff must surface every row instead of "Merkle trees are identical".
…cking markTableTruncated already leaves every leaf dirty, which is exactly the adaptive drain's escalated state. Record it in the escalator so UPDATEs landing on a re-populated table later in the same bounded drain skip per-PK tracking instead of re-running the threshold count and a redundant mark-all-dirty. Guarded by esc != nil (nil in continuous mode); INSERT/DELETE split-merge tracking is unaffected.
2e79ea7 to
49da06a
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (1)
tests/integration/mtree_truncate_test.go (1)
106-110: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueLog the failure of
os.Removeduring cleanup.Although this is a best-effort cleanup, explicitly handling and logging potential errors from
os.Removealigns with the convention used elsewhere in the test (likeMtreeTeardown) and ensures failures don't silently go unnoticed. Based on learnings, cleanup callbacks in integration tests should log expected failures witht.Logfrather than silently dropping them.(Note: Ignoring the error from
filepath.Globis safe here since it only returns an error for malformed patterns, and the hardcoded pattern is valid.)♻️ Proposed refactor
files, _ := filepath.Glob("*_diffs-*.json") for _, f := range files { - os.Remove(f) + if err := os.Remove(f); err != nil { + t.Logf("Warning: failed to remove diff file %s: %v", f, err) + } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/integration/mtree_truncate_test.go` around lines 106 - 110, Handle the error returned by os.Remove in the cleanup loop and log any failure with t.Logf, matching the existing MtreeTeardown cleanup convention while preserving best-effort cleanup. Keep the filepath.Glob error handling unchanged.Source: Learnings
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@tests/integration/mtree_truncate_test.go`:
- Around line 106-110: Handle the error returned by os.Remove in the cleanup
loop and log any failure with t.Logf, matching the existing MtreeTeardown
cleanup convention while preserving best-effort cleanup. Keep the filepath.Glob
error handling unchanged.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 4ff4d915-3f6d-49b8-a961-8e136e0a3252
📒 Files selected for processing (2)
internal/infra/cdc/listen.gotests/integration/mtree_truncate_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
- internal/infra/cdc/listen.go
The CDC drain's message switch only handled Insert/Update/Delete, so the TruncateMessage pgoutput publishes for a TRUNCATE (the publication is created with the default publish list, which includes truncate) was silently dropped. No leaves were dirtied, the truncated node's Merkle tree kept its pre-truncate hashes, and mtree table-diff reported the nodes as in sync.
A truncate carries only relation OIDs -- no per-row changes -- so per-PK tracking is impossible. React with the adaptive drain's bulk path instead: mark every leaf of each truncated relation dirty (cascades list all published relations) so the next tree update rehashes from the live, now-empty table. Buffered UPDATEs for the table are purged as at escalation time; a failed mark is terminal so a stale tree is never reported current. Applies to both bounded drains (table-diff/update) and continuous listen, which share the decode switch.
The integration test reproduces the field report: identical nodes, clean diff, TRUNCATE on n2, then a second diff must surface every row instead of "Merkle trees are identical".